1. Field of the Invention
The present invention relates to a device for detecting a motion vector used for motion compensating of motion pictures, and particularly relates to a motion vector detecting device for detecting motion vectors in accordance with a block matching method.
2. Description of the Background Art
For transmitting and storing image signals having a huge volume of data, a data compressing technique is indispensable for reducing the volume of data. Image data includes considerable redundancy caused, e.g., by a correlation between neighboring pixels and visual properties of humans. The data compression technique for suppressing the redundancy of image data to reduce the volume of data to be transmitted is called as a high efficiency coding. An inter-frame (inter-field) predictive coding method is one of such high efficiency coding methods. The following processing is executed in this inter-frame (inter-field) predictive coding method.
Calculation is performed for each pixel to obtain a prediction error, which is a difference between pixel data in a current frame (or field) to be coded and pixel data at the same position in a reference frame (or field) preceding or succeeding in time the current frame. The prediction error calculated is used in the coding process to be performed thereafter. According to this method, if the images contain less motion a prediction error value is small, and the coding can be performed efficiently, because a high correlation is present between the frames (fields). If the images contain large motion, however, the correlation between frames (fields) is small so that a large error occurs, resulting in increase in volume of data to be transmitted. An inter-frame (or inter-field) predictive coding method with motion compensation is proposed as a method for overcoming the above problem.
FIG. 50 schematically shows a structure of a predictive coding circuit in the prior art. In FIG. 50, the predictive coding circuit includes a motion compensation predictor 920 which detects a motion vector for image signals applied from a preprocessing circuit at an upstream stage to produce a motion-compensated reference image in accordance with the motion vector detected, a loop filter 922 which filters reference image pixel data read from motion compensation predictor 920, a subtractor 924 which obtains a difference between the input image signal and the output signal of loop filter 922, an orthogonal transformer 926 which performs an orthogonal transformation on the output signal (data) of subtractor 924, and a quantizer 928 which quantizes the output data of orthogonal transformer 926.
Motion compensation predictor 920 has a frame memory for storing image data of a preceding frame (or field), and produces the motion-compensated reference image pixel data in accordance with the image data of the preceding frame and input image signal data. The reference image pixel data motion-compensated and produced is stored in another buffer memory in motion compensation predictor 920. Loop filter 922 is provided for improving an image quantity.
Orthogonal transformer 926 performs an orthogonal transformation such as DCT transformation on the data received from subtractor 924 in a unit of a block of a predetermined size (usually 8xc3x978 pixels). Quantizer 920 quantizes the transformed pixel data.
Motion compensation predictor 920 and subtractor 924 perform the inter-frame prediction (or inter-field prediction) for motion compensation, and reduce the temporal redundancy of the motion image. The spacial redundancy in the motion image is reduced by the orthogonal transformation by orthogonal transformer 926.
The coding circuit further includes an inverse quantizer 930 for transforming the data quantized by quantizer 928 into the original signal state, an inverse orthogonal transformer 932 for performing inverse orthogonal transformation on the output data of inverse quantizer 930, and an adder 934 for adding the output data of loop filter 922 to the output data of inverse orthogonal transformer 932. The inverse quantizer 930 and the inverse orthogonal transformer 932 produce the image data to be used in the inter-frame (inter-field) prediction for a succeeding frame (or field). Thus, inverse orthogonal transformer produces a difference value code to be transmitted. Adder 934 adds the output data of loop filter 922 to the inter-frame (inter-field) difference data received from inverse orthogonal transformer 932, whereby the image data of the current frame (field) is reproduced. The output data of adder 934 is written into the frame buffer included in motion compensation predictor 920. Description will now be given on the manner of detecting a motion vector mv in motion compensation predictor 920.
It is now assumed that a picture 950 is formed of 352 dots (pixels)xc3x97288 rows as shown in FIG. 51. Picture 950 is divided into a plurality of blocks each including 16xc3x9716 pixels. The motion vectors are detected on a block-by-block basis. It is assumed that a search area representing an area, in which a motion vector is searched, is formed of an image block 956. This image block (search area) 956 is larger by 16 pixels than a block 955 in each of the horizontally opposite directions and vertically opposite directions on the screen. Block 954 is located in the same position as a target block (template block) 952. Template block 952 is the current image block. The motion vector is detected with respect to this template block 952. Searching of the motion vector is performed in the following manner.
In FIG. 51, a block indicated by a vector (i, j) has a displacement (i, j) with respect to template block 952. This vector (i, j) is a motion vector candidate. An estimation function value is obtained. This estimation function value is, for example, an absolute difference value sum (or squared difference sum) of each pixel data in template block 952 and each respective pixel (in the same position) in the block having the displacement vector (i, j). The operation of obtaining the estimation function values is executed on every displacement in a range of vectors (i, j) from (xe2x88x9216, xe2x88x9216) to (+16, +16). The estimation function values are obtained for all the blocks (prediction image blocks) of the image blocks (search window blocks) in search area 956, and the prediction image block having the minimum estimation function value is detected. The displacement relative to block 954 of the prediction image block having the minimum estimation function value is determined as the motion vector with respect to template block 952.
This detection of the motion vector is performed before calculating the prediction error. The prediction image is the frame (or field) preceding or succeeding in time the current frame (or field). In accordance with the calculated motion vector, the prediction image of the frame (or field) to be referred to is moved. The image data in the position shifted by the motion vector of the frame (or field) to be referred to is used as the reference image, and each pixel in this reference image is used as the predictive value. Calculation is performed to obtain each prediction error between the pixels in the same position of the reference frame (or field) after the movement and the current frame (or field), and the calculated prediction errors are transmitted together with the motion vector.
The current picture and the reference picture are divided into blocks, and the reference image block having the highest correlation with the reference image block is obtained. This method is referred to as the block matching method. According to this block matching method, the reference image block having the highest correlation in a unit of a pixel block can be detected so that the prediction error can be small, and coding with high efficiency can be achieved. This motion vector must be transmitted every pixel block. If the block size is small, the number of blocks is large so that the volume of information to be transmitted becomes large. If the block size is large, the motion detection cannot be performed effectively. Accordingly, the pixel block generally has a size of 16xc3x9716 pixels as described above.
If the search area for the motion vector is wide, the reference pixel block having a higher correlation can be detected so that the motion vector can be detected with higher accuracy. If the search area is widened, however, the motion vector candidates increase in number so that the time required for detecting the motion vector increases. In the case where one pixel block is formed of 16xc3x9716 pixels as already described, the absolute difference values of 256 pixels must be obtained, and then the sum of these values must be obtained. By the operation of obtaining the sum, the estimation value for one motion vector candidate is obtained. This estimation value is obtained for each block in the search area, and the reference image block providing the minimum value is obtained. Therefore, this method suffers from such a problem that fast detection of the motion vectors and therefore fast coding of the image data are impossible if the search area is widened for obtaining an optimum motion vector.
An object of the invention is to provide a motion vector detecting device which can rapidly detect motion vectors with high accuracy.
Another object of the invention is to provide a motion vector detecting device which can cope with search areas of different sizes without difficulty.
Still another object of the invention is to provide a motion vector detecting device, which can easily perform sub-sampling of input pixel data at different sub-sampling rates, to rapidly detect motion vectors.
A motion vector detecting device according to the invention includes an image data input circuit for receiving template block pixel data and search area pixel data, and performing sub-sampling on the received pixel data to produce an output, an operation circuit for receiving the template block pixel data and the search area pixel data from the pixel data input circuit, and calculating an estimation value representing a candidate of the motion vector according to a block matching method with respect to the template block, and a motion vector determining circuit for receiving the estimation value from the operation circuit, and determining the motion vector with respect to the template block in accordance with the received estimation values.
The operation circuit includes a shift register circuit including shift registers for transferring the pixel data in one direction, and performing transference and storage of the search area pixel data applied from the image data input circuit, and a plurality of element processors provided corresponding to the shift registers. Each of the element processors includes a data register circuit for storing template block pixel data applied from the image data input circuit, and a calculating circuit for receiving the storage data in the data register circuit and the storage data in the corresponding shift register, and performing predetermined arithmetic operation on the received storage data to produce and output a component of the estimation value. The data register circuits in the plurality of element processors store data of different pixels in same template block.
The motion vector detecting device according to the invention further includes a path setting circuit for changing a data transfer path of the shift register circuit in accordance with the sub-sampling rate information.
By changing the data transfer path of the shift register circuit in accordance with the sub-sampling rate, the sub-sampled search area pixel data can be stored in the shift registers corresponding to the element processors. In the element processor, sub-sampled template block pixel data is stored. At this time, the operation circuit stores the pixel data of the template blocks corresponding in number to the sub-sampling rate. Calculation of the estimation values for the template blocks corresponding in number the sub-sampling rate can be performed in parallel within the operation circuit, and the motion vector can be detected without increasing the operation processing time even if the search area is widened. Change of the data transfer path of the shift registers is merely performed, and a plurality of sub-sampling rates can be employed while suppressing increase in hardware volume. Since it is possible to employ a plurality of sub-sampling rates, optimum search areas for specific applications can be determined.